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EXECUTIVE  SUMMARY 


Recent  policy  changes  in  the  Department  of  Defense  have  resulted 
in  new  guidelines  and  considerations  to  the  service  components  in  the 
area  of  weapon  system  developmental  testing.  It  is  important  for  con- 
tractor program  managers  and  members  of  their  management  team  to  under- 
stand the  ramifications  of  this  guidance  in  order  to  "tailor"  a test 
program  that  will  be  responsive  to  the  intent  of  this  guidance. 

Comprehensive  and  integrated  test  planning,  together  with  sound 
test  engineering  methodology  and  errorless  test  operations,  are  factors 
a contractor  program  manager  must  control  in  order  to  assure  program 
success. 

This  paper  addresses  the  considerations  a contractor  System  Test 
Manager  must  make  during  the  Conceptual  Phase  of  an  Artiy  missile  system 
in  order  to  "tailor"  a test  program  that  will  provide  meaningful  evalu- 
ation of  deliverable  hardware  in  terms  of  technical  performance  within 
prescribed  schedule  and  cost  constraints. 

The  first  chapter  identifies  the  genesis  of  policy  direction  re- 
garding Army  development  testing  thru:  Department  of  Defense  Directive 
5000.1,  Acquisition  of  Major  Defense  Systems;  Department  of  Defense 
Directive  5000.3,  Test  and  Evaluation;  and  in  turn,  the  Army  Letter  of 
Instruction,  Implementing  the  New  Material  Guidelines,  and  then  sum- 
marizes the  most  important  aspects  of  these  documents  as  they  apply  to 
developmental  testing  and  contractor  participation  in  such  testing. 

The  second  chapter  focuses  on  some  of  the  major  considerations  a 
contractor  System  Test  Manager  should  make  during  the  Conceptual  Phase 


and  recommends  a modified  Systems  Engineering  approach  be  adopted  in 
the  test  planning  process  to  give  the  contractor  visability  to  the 
interactions  between  test  oriented  elements  and  all  other  program 
elements  thereby  introducing  a systematic  way  to  identify  and  control 
test  efforts,  cost,  and  cost  estimating. 

The  third  chapter  expands  the  sphere  of  basic  test  considerations 
and  explores  the  test  planning  that  will  be  required  for  follow-on  phases. 
Particular  emphasis  has  been  placed  on  Range  and  Range  Support  planning 
for  flight  test  programs. 
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Introduction 

Recent  policy  changes  in  the  Department  of  Defense  have  resulted 
in  new  guidelines  to  the  service  components  in  the  area  of  weapon  system 
development  testing.  It  is  important  for  contractor  program  managers 
and  members  of  their  management  teams  to  understand  the  ramifications  of 
this  guidance  in  order  to  "tailor"  a test  program  that  will  be  responsive 
to  the  intent  of  the  guidance, 
i Comprehensive  and  integrated  test  planning  together  with  sound  test 

engineering  methodology  and  errorless  test  operations  are  factors  a con- 
tractor program  manager  must  control  in  order  to  assure  program  success. 

' This  paper  addresses  the  considerations  a contractor  System  Test 

Manager  must  make  during  the  Conceptual  Phase  of  an  Army  missile  system 
in  order  to  "tailor"  a test  program  that  will  provide  meaningful  evalu- 
ation of  deliverable  hardware  in  terms  of  technical  performance  within 
prescribed  schedule  and  cost  constraints. 


CHAPTER  I 


SURVEY  OF  ARMY  TEST  POLICY  CHANGES 

The  genesis  of  policy  direction  regarding  development  testing  is 
found  in  Department  of  Defense  Directive  5^00,1,  Acquisition  of  Major 
Defense  Systems,  13  July  1971,  and  is  amplified  in  Department  of  Defense 
Directive  5000.3,  Test  and  Evaluation,  November  1972.  In  turn,  the  Army 
has  expanded  the  direction  and  passed  it  to  all  commands  through  a Letter 
of  Instruction  (LOI),  Implementing  the  New  Material  Guideline,  23  August  1972. 

Understanding  the  direction  and  intent  of  the  guidance  provided  in 
each  of  these  documents  is  essential  if  a contractor  is  to  design  a develop- 
ment test  program  that  is  responsive  to  the  guidance  found  in  the  above 
documents.  The  following  discussion  serves  to  familiarize  the  contractor 
System  Test  Manager  with  the  most  important  aspects  of  these  documents  as 
they  apply  to  Army  weapon  system  testing  and  contractor  participation  in 
this  activity. 

POD  5000.1  MAJOR  ACQUISITION  OF  DEFENSE  SYSTEMS 

DOD  5000.1  provides  the  following  overall  guidance  on  weapon  system 
program  considerations: 

1.  Practical  tradeoffs  shall  be  made  between  system  capability, 
cost  and  schedule. 

2.  Traceability  of  cost  estimate  shall  be  maintained. 

3.  Early  development  shall  be  devoted  to  only  those  parameters 
necessary  to  basic  design. 

4.  Programs  shall  be  structured  to  ensure  that  demonstration  of 
program  objectives  is  the  pacing  function. 
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5.  Technical  uncertainty  shall  be  continually  assessed.  Models 
and  the  like  will  be  used  to  increase  program  confidence. 

6.  Test  and  Evaluation  shall  begin  as  early  as  possible  making  use 


of  the  most  realistic  test  environment  possible. 

7.  Cost  type  prime  and  subcontractors  are  preferred  where  sub- 
stantial developmental  effort  is  required. 

8.  Source  selection  shall  be  based  on  the  contractor's  ability  to 
design,  build,  and  test  on  a timely  and  cost  effective  basis. 

9.  A realistic  work  breakdown  structure  (WBS)  shall  be  established 
to  provide  a framework  for  planning,  control,  progress  reporting, 
and  an  estimating  base  for  future  costs  of  the  program.  (1) 


DOD  5000.3  TEST  AND  EVALUATION 


DOD  5000.3  is  more  specific  than  DOD  5000.1  in  policy  for  the  conduct  of 
development  test  and  evaluation  by  the  DOD  components.  The  following  direct 
quote  clearly  identifies  where  the  emphasis  should  be  placed  during  pro- 
gram test  planning. 

1 . General 

a.  Test  and  evaluation  shall  commence  as  early  as  possible 
and  be  conducted  throughout  the  system  acquisition  pro- 
cess as  necessary  to  assist  in  progressively  reducing 
acquisition  risks  and  in  assessing  military  worth. 

b.  Acquisition  schedules  will  be  based,  interalia,  upon 
accomplishing  test  and  evaluation  milestones  prior  to 
the  time  that  key  decisions  which  would  commit  signifi- 
cant added  resources  are  to  be  made. 

c.  Before  the  initiation  of  development  of  a new  system, 
test  and  evaluation  using  existing  systems,  or  modifi- 
cations thereto,  may  be  appropriate  to  help  define  the 
military  need  for  the  proposed  new  system  and  to  estimate 
its  military  utility.  Determination  of  military  worth. 
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need,  and  utility  will  be  accomplished  in  accordance 
with  other  DOD  directives. 

d.  All  test  and  evaluation  activities  shall  consider 
environmental  issues  and  provide  assessments  for 
review  as  early  as  possible  in  the  test  planning 
cycle.  (See  DOD  Directive  6050.1) 

2.  Development  Test  and  Evaluation  (DT&E).  DT4E  is  that 
test  and  evaluation  conducted  to:  demonstrate  that  the 
engineering  design  and  development  process  is  complete; 
demonstrate  that  the  design  risks  have  been  minimized; 
demonstrate  that  the  system  will  meet  specifications; 
and  estimate  the  system's  military  utility  when  intro- 
duced. DT&E  is  planned,  conducted,  and  monitored  by  the 
developing  agency  of  the  DOD  Component,  and  the  results 
thereof  are  reported  by  that  agency  to  the  responsible 
Military  Service  Chief  or  Defense  Agency  Director. 

a.  DT&E  shall  be  started  as  early  in  the  development 
cycle  as  possible  and  include  testing  of  component(s 
subsystem(s) , and  prototype  or  preproduction  model (s 
of  the  entire  system.  Compatibility  and  interoper- 
ability with  existing  or  planned  equipments  and 
systems  shall  be  tested. 

b.  During  the  development  phase  following  the  Program 
Initiation  Decision  (Milestone  I),  adequate  DT&E 
shall  be  accomplished  to  demonstrate  that  technical 
risks  have  been  identified  and  that  solutions  are 
in  hand. 

c.  During  the  Full-Scale  Development  phase  and  prior 
to  the  first  major  production  decision,  the  DT&E 
accomplished  shall  be  adequate  to  insure:  that 
engineering  is  reasonably  complete;  that  all  signifi 
cant  design  problems  (including  compatibility,  inter 
operability,  reliability,  maintainability,  and 
logistical  considerations)  have  been  identified; 
and  that  solutions  to  the  above  problems  are  in  hand 

d.  For  those  systems  which  have  a natural  interface 
with  equipment  of  another  Component  or  may  be 
acquired  by  two  or  more  Components,  joint  DT&E  may 
be  required.  Such  joint  testing  will  include  par- 
ticipation and  support  by  all  affected  Components 
as  appropriate.  (2) 
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ARMY  LOI  IMPLEMENTING  THE  NEW  MATERIAL  GUIDANCE 

The  Ariny  LOI  describes  Development  Testing  (DT)  and  Operational  Testing  (OT) 
as  a function  of  the  life  cycle.  Figure  1,  Army  Weapon  Systems  Acquisition 
Life  Cycle,  depict  the  relationship  the  various  test  categories. 

1.  Development  Testing  (DT) 

DT  is  the  responsibility  of  the  Army  Materiel  Command  and  in- 
cludes engineering  testing  and  that  part  of  service  testing 
which  assesses  operability  and  maintainability  of  the  system 
by  the  prospective  user.  Initial  Production  Testing  (IPT)  is 
now  considered  as  included  within  the  third  phase  of  develop- 
ment testing. 

DT  is  to  be  started  early  in  the  development  cycle  as 
possible  and  should  first  test  components,  then  subsystems, 
and  finally  prototypes  or  preproduction  models  of  the  entire 
system.  Any  test  data  which  was  previously  acquired,  re- 
gardless of  the  source,  should  be  used  whenever  applicable. 
Representative  user  personnel  should  participate  in  this  testing 
in  order  to  insure  "soldier  proofing",  i.e.,  assure  that  human 
factor  considerations  have  been  properly  designed  into  the 
hardware.  The  DT  test  results,  reports,  and  evaluations  must 
be  distributed  so  that  review  by  commands  and  agencies  involved 
in  the  decision  making  process  can  be  accomplished  in  a timely 
manner. 

During  advanced  development,  adequate  DT  must  be  accom- 
plished to  demonstrate  that  the  technical  risks  have  been  fully 
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identified  and  are  manageable. 

During  engineering  development  and  prior  to  the  first  major 
production  decision,  the  DT  accomplished  should  be  adequate  to 
insure  that  the  engineering  is  reasonably  complete;  that  all 
significant  design  problems  (including  reliability,  maintain- 
ability and  logistical  considerations)  have  been  resolved;  that 
manufacturing  methods  and  production  engineering  data  have  been 
generated;  and  that  production  planning  has  been  completed  to 
the  extent  required  to  provide  a realistic  basis  for  estimating 
costs  and  delivery  schedules. 

DT  of  early  production  models  must  be  accomplished  to  assure 
that  the  characteristics  of  the  production  item  meet  the  pre- 
scribed system  specifications. 

a.  DT  I is  conducted  early  in  the  development  cycle  (Vali- 
dation Phase).  Components,  subsystems,  or  the  complete 
system  is  tested  to  determine  if  the  system  is  ready 
for  Full  Scale  Development.  If  this  is  a test  of  com- 
peting systems,  then  its  purpose  is  to  provide  a compari- 
son between  the  systems  being  tested.  Operational  Testing, 
if  appropriate,  is  conducted  concurrently  with  this  test 
phase. 

b.  DT  II  provides  the  technical  data  necessary  to  make  a 
decision  if  the  system  is  ready  for  production.  It 
measures  the  technical  performance  of  the  item,  its 
associated  tools,  test  equipment,  training  package,  and 
maintenance  support  package.  Reliability  and 


maintainability  are  also  evaluated  during  this  test 
phase.  DT  II  also  includes  "soldier  proofing"  by  repre- 
sentative user  personnel  but  not  necessarily  in  an 
operational  environment.  Operational  testing  is  cor- 
ducted  concurrently  by  the  OT  command  or  agency, 
c.  DT  III  is  conducted  on  systems  from  the  initial  produc- 
tion to  verify  that  the  hardware  meets  its  production 
specification.  This  also  confirms  that  any  deficiencies 
previously  found  have  been  ;corrected , and  this  replaces 
the  Initial  Production  Test.  (3) 

2.  Operational  Testing  (OT) 

OT,  like  DT,  is  started  as  early  as  practicable  in  the  develop- 
ment cycle,  beginning  with  early  prototypes  and  continuing 
through  production.  OT  is  accomplished  by  the  user  and  support 
personnel  of  the  type  and  qualifications  of  those  expected  to 
use  and  maintain  the  system  when  deployed.  The  actual  testing 
will  normally  be  conducted  in  phases,  keyed  to  the  appropriate 
decision  point.  The  OT  test  results,  reports,  and  evaluations 
must  be  distributed  in  a very  timely  manner  so  that  the  needed 
review  by  the  decision  makers  can  be  accomplished  without  delay. 

The  Army  has  designated  that  the  Operational  Test  and 
Evaluation  Agency  (OTEA)  is  responsible  for  making  sure  that 
adequate  OT  is  conducted  for  all  major  systems.  The  OTEA  will 
coordinate  with  the  user  in  planning  for  and  the  conduction  of 
OT.  OTEA  prepares  the  independent  evaluation  of  the  adequacy 
of  the  testing  and  the  validity  of  the  results  at  the  completion 
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of  each  phase  of  OT. 


a.  OT  I provides  early  information  as  to  the  operational 
suitability  of  the  system  and  also  provides  a comparison 
to  existing  systems.  This  test  phase  can  identify 
problems  which  would  be  addressed  in  subsequent  testing. 
If  competitive  prototypes  are  being  tested,  then  the 
testing  would  be  accomplished  with  DT  I using  one  co- 
ordinated test  plan. 

b.  OT  II  is  done  just  prior  to  the  production  decision  and 
it  provides  a specific  assessment  of  the  system's  opera- 
tional suitability  and  effectiveness.  It  also  provides 
the  data  necessary  to  check  on  and  modify  if  necessary 
the  training,  logistics,  organizational,  and  employment 
concepts.  A small  troop  is  used  in  this  phase  to  submit 
the  equipment  to  a realistic  operational  environment. 

Data  from  this  phase,  together  with  an  independent  evalu- 
ation, is  required  by  the  decision  makers  to  assist  them 
in  making  a Low  Rate  Initial  Production  decision. 

c.  OT  III  is  accomplished  using  early  production  models 
and  provides  the  following: 

(1)  Information  pertaining  to  earlier  estimates  of 
operational  effectiveness. 

(2)  Determines  the  operational  suitability  of  the 
production  model. 

(3)  Optimizes  organization  and  doctrine. 

(4)  Validates  training  and  logistic  requirements. 

i 

i 8 


(5)  Identifies  actions  that  need  to  be  taken  before 
deployment.  (4) 

WEAPON  SYSTEM  LIFE  CYCLE 

As  additional  background  to  assist  the  contractor  in  test  planning 
activities,  it  is  important  that  he  have  an  understanding  of  the  evolution 
of  the  life  cycle  of  a weapon  system.  The  evolution  of  a weapon  system 
follows  a series  of  well-defined  phases  which  together  form  the  system 
life  cycle.  In  each  phase  there  is  a sequence  of  activities,  events  and 
specific  outputs  germane  to  that  phase  which  must  be  accomplished  before 
moving  to  the  next  phase.  The  following  paragraphs  on  System  Life  Cycle 
define  these  phases  and  presents  a generalized  sequence  of  engineering 
activities  accomplished  in  each  phase. 

1.  Conceptual  Phase.  During  this  phase,  the  technical,  military, 
and  economic  bases  for  an  acquisition  program  are  established 
through  comprehensive  systems  studies  and  experimental  hardware 
development  and  evaluation.  The  conceptual  phase  is  highly 
iterative.  Its  stages  overlap  rather  than  occurring  sequentially; 
however,  flowing  from  interacting  inputs  of  operational  needs  and 
technology,  generally  the  following  stages  occur: 

a.  Identification  and  definition  of  conceptual  systems. 

b.  Analysis  (threat,  mission,  feasibility,  risk,  cost, 
tradeoffs,  etc.). 

c.  Experimentation  and  test  (of  operational  requirements,  key 
components,  critical  subsystems  and  marginal  technology). 

The  outputs  of  the  Conceptual  Phase  are  alternative  systems 
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(-■c’udlr.g  a preferred  system)  and  their  associated  program 
charA'-t.erisT.ics  (costs,  schedules,  and  operational  parameters) 

on  ; combination  of  analyses,  experiments  and  test  results. 
Validation  Phase.  This  is  the  phase  in  which  the  major  program 
characteristics  (technical,  cost  and  schedule),  through  exten- 
sive analysis  and  hardware  development,  are  validated  and  is 
often  identified  with  Advanced  Development.  It  is  preferred 
to  rely  on  hardware  development  and  evaluation  rather  than  paper 
studies,  since  this  provides  a better  definition  of  program 
characteristics,  higher  confidence  that  risks  have  been  resolved 
or  minimized  and  greater  confidence  in  the  ultimate  outcome. 

In  an  idealized  case,  this  phase  ends  when  a representabV 
engineerir.'  model  has  been  demonstrated  successfully. 

Full-Scale  Development  Phase.  Duri-g  this  phase,  the  weapon 
system  including  all  of  the  items  necessary  for  its  support 
(training  equipment,  maintenance  equipment,  handbooks  for  opera- 
tion and  maintenance,  etc.)  is  designed,  fabricated  and  tested. 

The  ’ntended  '^utput  is  a "hardware  model"  whose  perfom^ance 
and  pliability  has  been  proven  experimentally  and  the  documen- 
tation needed  to  produce  for  inventory  use.  An  essential  activity 
of  the  development  phase  is  test  and  evaluation,  both  that  con- 
ducted by  contractors  and  that  conducted  by  the  Se-^vice. 

Prody  tion  Phase.  During  this  phase  the  weapon  system,  including 
training  equipment,  spares,  etc.,  is  produced  for  operational  use. 
Deployment  Phase.  During  this  phase  the  weapon  system  is  pro- 
vided to  and  used  by  operational  units. 


CHAPTER  II 


CONSIDERATIONS,  GUIDANCE  AND  APPROACH  FOR 
CONTRACTOR  TEST  PLANNING  DURING  THE  CONCEPTUAL  PHASE 

This  chapter  focuses  on  the  considerations  a contractor  should 
make  during  the  Conceptual  Phase  (CP)  of  an  Army  missile  system  to 
assure  adequate  evaluation  of  technical  performance  for  deliverable 
hardware  within  prescribed  cost  and  schedule  constraints. 

Comprehensive  test  planning  is  the  tool  a contractor  has  at  his 
disposal  to  accomplish  this  task.  By  and  large,  test  planning  is  an 
artform  that  heretofore  has  been  "fuzzy"  and,  consequently,  costly.  New 
approaches  to  test  planning  must  be  explored  if  cost  effectiveness  is 
ever  to  be  introduced  in  test  process. 

In  order  to  give  the  contractor  visibility  to  the  interactions 
between  test  oriented  program  elements  and  all  other  program  elements  in 
identifying  and  controlling  test  efforts,  cost  and  cost  estimating,  this 
report  recommends  that  a modified  systems  engineering  approach  be  adopted 
in  the  contractor  test  planning  process.  This  can  be  achieved  by  dia- 
gramming the  test  related  program  elements  and  their  interactions  with 
other  program  elements.  Figure  2 illustrates  the  concept.  Task  oriented 
block  diagrams  can  be  constructed  for  each  phase  of  the  life  cycle  and 
can  depict  the  major  test  related  tasks  and  activities  to  be  accomplished. 
These  tasks  and  the  general  sequence  in  which  they  are  to  be  accomplished 
can  be  displayed  on  an  activity  flow  diagram.  The  concept  of  iterative 
refinement  or  "tailoring"  can  be  handled  through  the  use  of  closed  loop 
notation.  Major  blocks  on  the  diagram  can  then  be  further  expanded  or 


11 


described  by  accompanying  text. 

In  order  to  demonstrate  the  approach,  this  paper  addresses  test 
activities  associated  with  a hypothetical  Conceptual  Phase.  The  Con- 
ceptual Phase  was  selected  due  to  the  amount  of  strategic  test  planning 
and  tradeoffs  that  occur  in  this  phase  which  establishes  a test  "bench- 
mark" for  follow-on  phases. 

Since  discussions  will  be  directed  to  addressing  the  test  related 
activities  and  tradeoff  candidates  in  the  Conceptual  Phase  (CP),  an  ex- 
pansion of  the  contractor  responsible  efforts  during  CP  is  in  order. 

The  efforts  during  CP  are  concerned  primarily  with  comprehensive 
studies  leading  to  the  technical,  economic  and  military  basis  for  a 
decision  to  develop  a new  capability.  Specifically,  it  involves  the 
development  and  selection  of  the  best  system  approach  which  will  satisfy 
established  mission  and  performance  requirements,  and  the  investigation 
of  feasibility  of  the  approach  from  a technical,  cost,  and  schedule 
standpoint.  Some  of  the  activities  which  characterize  this  period  in- 
clude the  preparation  of  a preliminary  system  specification,  a system 
development  plan,  a maintenance  support  plan,  a coordinated  test  plan, 
the  documentation  associated  with  the  prerequisites  to  contract  definition, 
a plan  for  conducting  contract  definition,  and  a description  of  the  system 
to  be  developed  (i.e.,  functional  baseline  documentation).  Near  the  end 
of  CP,  a system  review  is  conducted  by  both  the  contractor  program  office 
and  the  government  program  office  to  ensure  that  the  necessary  preliminary 
work  has  been  done  and  that  the  threat  and  operational  analyses,  the  trade- 
off and  cost  effectiveness  studies,  and  the  development  of  components  and 
technology  provides  a firm  foundation  for  engineering  development. 
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C.1 


CONTRACTOR  SYSTEM  TEST  RELATED  ACTIVITI 
CONCEPTUAL  PHASE  (CP) 

FIGURE  2 
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C.8 


C.ll 


SYSTEM  TEST  RELATED  ACTIVITIES 
INCEPTUAL  PHASE  (CP) 


FIGURE  2 


CP  ends  with  a customer  decision  to  pursue  the  program  and  to 
secure  the  necessary  approvals  to  proceed  into  validation  phase  (VP). 
Most  of  the  major  test  related  events  and  activities  of  this  phase  of 


the  life  cycle  are  described  by  Figure  2. 

The  remainder  of  this  chapter's  discussion  will  be  centered  on 
describing  major  test  related  activities  in  CP  as  depicted  by  the  coded 
blocks  in  Figure  2.  Since  system  test  activities  are  minimal  in  blocks 
C.l  through  C.6,  the  discussion  will  begin  with  block  C.7. 

BLOCK  C.7  IDENTIFICATION  OF  TEST  SUPPORT  ELEMENTS 

The  CP  RFP  will  be  utilized  to  define  the  test  support  requirements 
and  conduct  preliminary  design  synthesis  of  hardware,  software,  computer 
programs  and  personnel  required  for  the  validation  phase  test  program. 

The  elements  engineered  will  be  in  support  of  development,  qualification, 
maintainability,  reliability,  environmental,  field  and  other  required 
tests.  It  should  be  noted  that  specialized  equipment  and  software  de- 
veloped may  be  carried  over  into  the  FSD  phase  to  support  manufacturing 
operations  and  flight  testing. 

This  phase  will  establish  firm  test  support  end  item  and  component 
design  requirement  sheets  for  all  support  elements  (including  software 
and  computer  programs).  In  addition,  detail  development,  environmental, 
certification  and  qualification  test  programs  conducted  in  the  laboratory 
and  the  field  will  be  drawn  up  with  the  necessary  plans  to  achieve  these 
program  efforts.  The  level  of  effort  accomplished  will  be  to  the  detail 
necessary  to  provide  a firm  contractor  estimate  for  the  price  of  all 
testing  and  support  necessary  for  the  validation  phase  (VP). 
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BLOCK  C.7.1  IDENTIFICATION  OF  TEST  SUPPORT  REQUIREMENTS 


I 


The  test  activity  requirements  and  constraints  will  be  identified 
from  CP  specifications  and  also  from  test  plans  and  descriptions  in  the 
RFP.  In  addition,  the  requirements,  configuration  and  constraints  of 
existing  facilities  and  equipment  that  will  be  utilized  by  the  program 
will  be  factored  into  this  process  within  the  appropriate  function. 

BLOCK  C.7.2  PERFORM  TEST  SUPPORT  FUNCTIONAL  ANALYSIS 

A detailed  analysis  and  survey  will  be  performed  to  determine  the 
test  support  functions.  The  analysis  will  utilize  the  preliminary  infor- 
mation developed  in  pre-CP  activities  and  revised  for  the  CP  proposal 
effort.  During  this  phase,  the  analysis  will  be  revised,  refined  and 
significantly  expanded  to  completely  describe  all  test  elements.  It 
should  be  noted  that  all  reasonable  alternatives  will  be  developed  and 
included  in  the  analysis.  Test  support  functions  to  the  lowest  indentured 
level  of  WBS  hardware  and  software  functions  are  required  to  comprehen- 
sively identify  the  criteria  necessary  for  the  development  of  design  re- 
quirements and  synthesis  of  elements.  The  analysis  expansion  should 
identify  and  define  all  test  support  elements,  and  should  establish  that, 
as  a minimum,  the  following  items  have  been  considered. 

1.  Functional  checkout  equipment  and  consoles 

2.  Environmental  simulation  equipment 

3.  Test  fixtures 

4.  Models  and  boilerplate  operations  equipment 

5.  Facilities  (mechanical,  civil,  electrical,  fluid,  gaseous, 
environmental,  communications). 
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6.  Operations  equipment,  instrumentation,  telemetry  and  land 
line  and  receiving  elements 

7.  Operation  range  support  (mechanical,  civil,  communications, 
electrical,  fluid,  gaseous,  range  safety,  photographic  vehicles, 
telemetry  station,  land  line  instrumentation) 

8.  Data  acquisition  and  reduction  equipment 

9.  Computer  programs 

10.  Procedures  and  other  software 

11.  Personnel  for  test  support  operation 

If  necessary,  the  expansion  will  be  to  the  level  where  there  is  identifi- 
cation and  definition  of  critical  and  long  lead  elements.  Iteration 
should  be  used  to  accomplish  the  detail  expansion  of  functions  and  assign- 
ment of  each  function  to  a Test  Support  element.  All  alternative 
approaches  to  accomplish  the  test  support  functions  necessary  for  VP  shall 
be  resolved  prior  to  the  end  of  CP.  Provisions  will  be  built  into  the 
analysis  to  insert  added  test  support  functions  and  element  assignments 
during  the  VP  phase  where  detail  design  makes  these  functions  apparent. 

BLOCK  C.7.3  ITFRATE  TEST  SUPPORT  ANALYSIS 

By  iterative  process  and  in  parallel  with  the  function  analysis. 
Engineering  will  refine  and  expand  the  hardware/software  approach  in- 
cluding developing  alternatives  to  accomplish  the  test  support  function. 
This  will  include  revising  previously  developed  schematic  diagrams, 
sketches  and  text  descriptions  as  proposed  during  the  VP  proposal  effort. 
In  addition  to  analyzing  hardware  and  facilities,  equal  emphasis  will  be 
placed  on  identifying  procedural  data  and  personnel. 


Preliminary  design  of  the  equipment,  software  and  facilities  will 
be  started  and  carried  to  sufficient  depth  to  clearly  show  the  design 


approach  to  each  element.  Studies  should  be  performed  to  determined  man- 
machine  interfaces  and  establish  the  personnel  required. 

BLOCK  C.7.4  TEST  SUPPORT  EVALUATION  AND  DECISION 

Test  support  elements  will  be  selected  primarily  by  using  quanti- 
tative measures  of  effectiveness  with  no  provisions  for  weighing  the 
various  effectiveness  parameters.  The  effectiveness  criteria  under  these 
conditions  are  usually  described  as  best  performance  for  lowest  cost  and 
best  schedules  with  little  attention  given  to  other  more  or  less  insig- 
nificant variables.  Typical  examples  that  fall  into  this  category  are: 

1.  Small  test  fixtures  and  jigs 

2.  Piece  part  instrumentation  both  for  laboratory  and  field 

3.  Calibration  equipment  and  tools 

4.  Software  and  computer  programs 

5.  Small  test  specimens 

6.  Relatively  simple  electronic,  structural  and  hydraulic 
operating  equipment 

However,  with  large  test  support  systems  containing  a multitude  of  elements 
the  variables  such  as  power,  size,  weight,  reusability,  application  to 
other  programs,  human  factors,  reliability,  safety,  etc.,  become  signifi- 
cant as  to  the  determination  of  the  most  effective  candidate.  In  this 
instance,  a complete  "Systems  Effectiveness"  model  will  be  assembled,  and 
utilize  both  quantitative  measures  of  effectiveness  and  weighting  criteria 
to  arrive  at  the  most  effective  system  to  satisfy  program  requirements. 
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Typical  examples  to  which  this  will  be  applied  are: 

1.  Test  stand  complex  (field) 

2.  System  test  laboratory 

3.  Facilities 

4.  Environmental  simulation  equipment 

5.  Data  acquisition  and  reduction  equipment. 

In  effect,  all  test  support  elements  will  be  selected  by  appropriate 
trade  studies  that  illustrate  and  verify  through  proper  evaluation  the 
candidate  selected  is  optimum  from  all  alternative  candidates  studied. 

It  should  be  noted  that  no  matter  how  trivial  a trade  may  be,  it  will 
always  be  documented  for  visibility  and  traceability  of  requirements, 
alternate  solution  evaluation  and  selection  decision. 

The  trade  studies  performed  in  this  phase  should  identify  those 
elements  that  show  substantial  risk  due  to  "state  of  the  art"  design, 
time  or  cost. 

BLOCK  C.8  INTEGRATED  TEST  PLAN 

The  integrated  test  plan  is  designed  to  show  the  coordinated  and 
integrated  program  of  all  test  to  adequately  demonstrate  technical  per- 
formance of  equipment  and  procedures  from  piece  part  through  complete 
systems  when  implemented  in  the  follow-on  phases.  The  categories  of  tests 
that  will  be  planned  are  Developmental,  Certification,  Qualification, 
System  and  Field  Operation  Test.  Typical  items  to  be  included  in  the  plan 
are  schedules;  time  allocation  for  procedure  development;  laboratory 
equipment  usage;  field  or  special  test  area  fixture  usage;  design,  fabri- 
cation and  procurement  of  equipment;  instrumentation  and  fixtures;  and 
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data  acquisition,  reduction  and  analysis. 


1.  Developmental  Tests.  These  are  tests  performed  to  demonstrate 
the  initial  design  performance  of  circuits,  structures,  mecha- 
nisms, aerodynamic  shape,  etc.,  to  show  hardware  feasibility. 

These  tests  include: 

a.  Breadboard  (electrical,  hydraulic,  pneumatic) 

b.  Mock-up  (structure,  system  and  subsystem) 

c.  Structural  test  (static  and  dynamic) 

d.  Limited  environmental  test  on  piece  parts 

e.  Functional  test  on  piece  parts  and  small  assemblies 

f.  Life  test  (piece  parts) 

g.  Model  test  (aerodynamics,  radiation,  R.F.). 

These  tests  assist  in  evaluating  the  design  and  verifying  that 
it  is  ready  for  limited  developmental  production. 

2.  Certification  Tests.  These  tests  are  performed  to  demonstrate 

1 

I that  the  design  is  ready  for  limited  field  performance  test  and 

t to  demonstrate  operational  capability.  The  tests  are  normally 

, conducted  in  limited  quantities.  These  tests  include: 

a.  Environmental  (limited) 

b.  Assembly  level  functional 

c.  System  static  and  dynamic  loads 

d.  Subsystem  functional 

e.  Limited  life  and  fatigue  test. 

3.  System  Tests.  These  tests  are  to  demonstrate  systems  hardware 

operating  compatibility  and  its  susceptability  to  external  in- 
fluences. These  tests  are  conducted  just  prior  to  first  field  1 

1 
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operations  tests  to  ensure  the  system  will  perform  under 
limited  specified  operating  conditions.  The  tests  include: 

a.  Laboratory  marriage 

b.  Static  firing  tests 

c.  Ground  operation  tests 

d.  Limited  environmental  tests 

e.  Selected  critical  field  operation  tests 

f.  System  instrumentation  checkout  and  verification. 

4.  Field  Operations  Test.  These  tests  are  conducted  using  complete 
prototype  systems  to  demonstrate  performance  capability  under 
limited  field  conditions.  These  tests  include: 

a.  Flight  test  (limited  configuration) 

b.  Ground  test 

c.  Environmental 

d.  Operations  and  maintenance 

5.  Qualification  Tests  conclude  the  testing  performed  on  systems 
and  subsystems  by  verifying  that  the  hardware  will  operate  at 
prescribed  performance  levels  under  all  environmental  and 
service  usage  conditions.  These  tests  include: 

a.  Environmental  (all  hardware  levels) 

b.  Reliability 

c.  System  flight  test  (full  configuration) 

d.  System  ground  (functional  and  environmental) 

e.  Maintenance  verification 

f.  Operation  verification. 
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BLOCK  C.9  PREPARE  ENGINEERING  SCHEDULES 
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Schedules  will  be  prepared  for  the  validation  phase  "cost"  package 
for  all  operations,  maintenance,  test  support,  production  and  deployment 
elements,  "^hey  will  be  based  on  the  system  specification,  subsystem  re- 
quirements, component  requirements,  preliminary  detail  design  and  customer 
master  schedule  of  activities.  The  prepared  schedules  will  indicate  as 
a minimum  the  following  elements  with  their  start,  significant  inter- 
mediate and  completion  dates. 

1.  Preliminary  design 

2.  Breadboard,  mockup  and  model  design  and  test 

3.  Basic  design 

4.  Functional  and  environmental  test 

5.  Certification  test 

6.  Qualification  test 

7.  Field  and  customer  demonstration  test 

8.  Reviews 

9.  Critical  hardware  milestones  of  design,  procurement  and  test 

10.  Soft  and  hard  tooling  design,  procurement,  fabrication,  check- 
out and  availability 

11.  Software  elements  for  maintenance,  production,  deployment 

12.  Test  support  elements  design,  procurement,  fabrication  checkout 
and  availability 

13.  Test  procedures 

14.  Hardware  procurement,  fabrication  and  availability 

15.  Personnel  training  and  demonstration 

16.  Facility  design,  checkout  and  availability. 
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All  pertinent  dates  and  span  times  concerning  significant  events 


will  be  included  in  the  schedules,  examples  of  which  are:  checking,  long 
lead  items,  data  to  the  customer,  customer  participation,  data  from  sub- 
contractors, associates  and  prime,  incentive  milestones,  engineering 
analysis  (stress,  weight,  circuits,  dynamics,  system,  etc.)  wind  tunnel, 
data  acquisition  from  test,  computer  simulation,  etc. 

BLOCK  C.ll  WORK  BREAKDOWN  STRUCTURE  UPDATED 


The  test  support  elements  identified  during  CP  provide  updating 
information  to  be  used  in  detailing  the  Work  Breakdown  Structure  for  the 
validation  phase.  Initially,  the  Work  Breakdown  Structure  is  furnished 
by  the  customer  and  expanded  by  him  to  the  summary  level.  Preliminary 
design  accomplished  during  CP  will  expand  WBS  levels  necessary  to  iden- 
tify discrete  work  packages  from  which  a firm  price  estimate  on  each 
detail  work  package.  Accompanying  each  work  package  will  be  a detailed 
shopping  list  of  assemblies,  parts,  itemized  activities  and  other  program 
elements  that  make  up  a work  breakdown  package. 

BLOCK  C.13  REVIEW  VP  PROPOSAL 

The  validation  phase  proposal  package  will  be  carefully  reviewed  and 
critiqued  through  a series  of  technical  and  management  reviews  culminating 
in  a final  contractor  "in-house"  management  review.  Test  participation  in 
these  reviews  is  mandatory.  The  purpose  of  these  reviews  is  to  assure 
correctness  of  all  technical  and  management  data;  depth,  completeness  and 
clarity  of  presentation;  and  that  proper  emphasis  has  been  placed  on  all 
areas  considered  critical  to  winning  validation  phase  contract.  The  reviews 
must  be  conducted  early  enough  to  allow  for  corrections  and  modifications. 
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CHAPTER  III 


ADDITIONAL  TEST  PLANNING  CONSIDERATIONS 

The  previous  chapter  presented  a generalized  approach  for  planning 
program  test  activities  during  the  Conceptual  Phase.  As  a result,  a 
"shopping  list"  of  possible  test  elements  was  identified  for  potential 
tradeoff  candidates  that  could  be  used  in  "tailoring"  the  development 
test  program. 

The  purpose  of  this  chapter  is  to  expand  the  sphere  of  considerations 
and  provide  the  contractor  with  additional  insight  as  to  the  level  of  de- 
tail that  will  be  required  for  test  planning  during  follow-on  phases.  The 
approach  used  is  to  consider  a few  test  elements  common  to  all  Army  missile 
systems;  such  as  environmental  tests,  marriage  tests,  qualification  tests, 
and  flight  tests.  Particular  emphasis  has  been  placed  on  Range  and  Range 
Support  planning  for  flight  test  programs.  It  has  been  the  author's  ex- 
perience that  unless  thorough  planning  and  coordination  is  effected,  pro- 
gram costs  are  likely  to  increase  and  schedules  will  be  jeopardized. 

PLANNING  ENVIRONMENTAL  TESTS 

Selected  environmental  test  (often  called  preflight  certification 
test)  will  be  planned  to  assure  the  system  and  subsystem  elements  can 
sustain  the  rigors  of  field  operation  and  demonstration  without  degrada- 
tion of  designed  performance.  These  tests  usually  include  most  severe 
dynamic,  static,  and  thermal  environments  as  well  as  radiation  (nucle?r 
and  electromagnetic)  susceptability  tests.  Specific  test  may  be,  but  are 
not  limited  to,  shock,  vibration,  acceleration,  temperature,  static  load. 


RFI-EMI  susceptability,  ganmaneutron  radiation  and  other  environments 
that  cause  significantly  high  level  mechanical  stress  or  electronic 
circuit  disturbance. 

If  there  is  low  confidence  that  section  level  hardware  does  not 
process  adequate  repeatability  of  performance  under  the  influence  of 
environments  resulting  from  manufacturing  processes  and  workmanship, 
engineering  design  problems,  etc.,  there  will  be  a Production  Environ- 
mental Test  (PET)  program  set  up  and  conducted  on  the  critical  items  on 
either  a sample  or  an  each  item  basis.  These  tests  will  be  performed 
using  selected  critical  environments  and  at  selected  levels  which  will 
not  degrade  performance,  but  will  show  up  deficiencies,  problems  or 
malfunctions  that  would  seriously  jeopardize  engineering  and  qualification 
testing. 

The  sequence  of  environmental  tests  will  be  governed  by  predeter- 
mined test  requirements  and  incorporated  in  an  applicable  test  plan.  The 
plan  will  be  updated  as  prototype  design  progresses.  Procedures  for  con- 
ducting the  tests,  and  the  necessary  support  equipment  used  in  the  perfor- 
mance of  the  tests  will  begin  with  the  establishment  of  preliminary  design 
updated  as  required.  The  results  of  the  tests  will  be  documented  in 
reports  that  fully  described  the  tests  as  to  conduct,  objectives,  accep- 
tance criteria,  failure  criteria,  events,  observables,  data  results  and 
conclusions.  Procedures  and  test  support  documents  will  be  referenced  in 
the  report  and  corrected  as  necessary  to  reflect  test  amendments,  correc- 
tions and  deviations.  The  documentation  developed  and  published  will  be 
placed  on  file  to  provide  a method  of  rapid  retrieval  for  future  reference. 
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PLANNING  MARRIAGE  TESTS 


The  marriage  tests  (also  called  System  Tests)  are  generally  planned 
to  be  conducted  on  two  or  more  assembled  units  of  a complex  subsystem, 
the  subsystem  itself  or  the  complete  system  to  determine  functional  com- 
patibility, interface  compatibility,  dynamic  range  of  all  functions  and 
other  systems  test  to  verify  design,  fabrication  and  specified  operational 
characteristics.  This  test  can  be  used  for  "proofing"  factory  acceptance 
test  procedures.  Emphasis  of  these  tests  will  shift  from  gaining  labora- 
tory engineering  knowledge  to  assuring  each  system  manufactured  is  ready 
for  flight  test.  It  is  mandatory  that  all  flight  units,  including  the 
instrumentation  packages  and  test  support  equipment,  be  assembled  and 
checked  out  in  this  manner  to  achieve  a high  confidence  level  in  attaining 
demonstration  test  goals. 

The  test  plans,  for  the  sequence  and  control  of  the  test,  procedure 
preparation,  design  and  fabrication  of  test  support  equipment,  and  docu- 
mentation preparation,  control  and  retrieval,  should  be  taken  into  con- 
sideration. 

PLANNING  QUALIFICATION  TESTS 

Qualification  tests  are  planned  to  demonstrate  that  the  developed 
system/ subsystem  will  operate  as  specified  and  there  is  no  out-of- tolerance 
performance  due  to  environmental  influence.  These  tests  are  conducted  on 
the  lowest  indentured  identifiable  assembly  or  component  on  the  WBS  that 
is  performance  critical  to  the  system  and  on  all  subsystems  as  required 
to  show  proof  of  performance.  The  types  of  test  that  may  be  performed  are: 
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1.  Laboratory  environmental  test  on  components,  assemblies,  sub- 
system and  system. 

2.  Field  Test  on  complete  system. 

Laboratory  Environmental  Test:  These  tests  will  subject  qualification 

hardware  to  simulated  environmental  stress,  corrosion  and  erosion  levels. 
The  object  of  these  tests  are  to  perform  parametric  performance  studies 
under  the  influence  of  the  specified  environmental  extremes,  including 
those  particular  levels  that  critically  disturb  the  test  specimen  per- 
formance. All  design  or  functional  deficiencies  observed  will  be  cor- 
rected by  an  appropriate  change  and  the  unit  retested.  The  test  results 
of  each  test  specimen  will  be  published  in  a report  that  contains  all 
pertinent  test  information  such  as: 

a.  Specimen  configuration 

b.  Specimen  hardware  changes  required 

c.  Support  equipment  and  configuration 

d.  Procedures  and  deviations 

e.  Data  (raw  and  reduced) 

f.  Design  and  performance  irregularities  observed 

g.  Results  and  analysis 

h.  Conclusions  and  recommendations 

i.  Test  configuration  and  deviations 

j.  Acceptance  and  failure  criteria. 

PLANNING  FLIGHT  TESTS 


These  tests  require  operation  of  the  system  under  actual  field 
environments  to  determine  the  system  performance  and  functional  capability 


within  specified  design  tolerances  and  requirements. 


The  test  results  of  each  system  test  will  be  published  in  a report 
that  contains  all  the  pertinent  data  described  in  the  laboratory  environ- 
mental test  paragraph. 

At  the  end  of  the  Development  Testing,  an  overall  report  will  be 
published  which  summarized  the  equipment  tested,  equipment  major  changes, 
confidence  level  of  the  test,  overall  test  results,  overall  conclusions 
and  recommendations. 

Another  salient  feature  of  field  test  is  that  it  provides  an  ex- 
cellent opportunity  to  verify  and  also  gather  valuable  background  material 
on  the  logistic  elements  (technical  publications,  personnel  utilization, 
maintenance  procedures,  material  allocation,  equipment  utilization,  train- 
ing required,  test  and  checkout  of  field  located  equipment,  etc.). 

Structuring  the  flight  test  program  is  a delicate  balance  of  art,  | 

: i 

science  and  political  procedures.  It  is  at  the  focal  point  of  leverage 
for  program  cost,  schedule  and  performance  considerations.  The  number 
of  missiles  to  be  fired,  the  intended  mission,  the  success  or  failure 

‘ j 

criteria,  the  types  of  targets,  etc.,  all  play  an  important  role  in  flight  i| 
test  planning.  | 

RANGE  FACILITIES  PLANNING  | 

Most  flight  test  programs  are  conducted  at  off  site  locations  due  M 

to  hazardous  nature  of  missile  launch  operations.  Generally  these  off-  Ij 

i 

site  locations  are  test  ranges  operated  by  the  government  or  in  a few  1] 

instances,  by  private  contracts.  I 

The  program  using  these  ranges  will  require  facilities  ranging  I 
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I from  limited  office  and  work  spaces  to  elaborate  launch  complexes.  The 

I modifying,  designing,  building,  and  operating  these  facilities  must  be 

' included  during  the  early  planning  efforts. 

In  most  cases  the  test  range  to  be  used  will  be  specified  by  the 
contracting  agency.  In  some  cases,  however,  it  is  necessary  to  conduct 
a survey  of  available  ranges  to  determine  which  best  satisfies  the  objec- 
tives of  the  test  program  at  minimum  cost  to  the  overall  program.  Selec- 
tion of  a facility  site  on  a given  range  is  a function  of  several  variables. 
These  include  the  availability  of  existing  facilities  which  may  be  suit- 
ably modified;  constraints  on  location  due  to  some  hazardous  nature  of 
the  system  during  ground  handling  or  in  flight;  system  performance 
characteristics  such  as  range,  field  or  fire  or  flight  envelope;  and  a 
test  design  requirement  such  as  maximizing  the  effectiveness  of  available 
instrumentation.  Once  the  range  is  chosen  and  a site  selected,  the  re- 
quired test  facilities  to  support  the  program  must  be  identified.  Generally 
flight  test  facilities  fall  into  the  following  categories: 

1.  Launch  complexes 

2.  Assembly  areas 

3.  Office  and  shop  work  areas 

4.  Storage  facilities  (warehouse  and  stockroom) 

5.  Special  utility  requirements 

The  test  facilities  may  be  supplied  by  the  contractor,  range,  or  by 
various  sub  and/or  associated  contractors  or  by  a mixture  of  all  of  these. 
Extensive  liaison  is  required  to  coordinate  these  agencies  and  the  co- 
ordination effort  must  be  included  in  planning  considerations  for  the 
flight  test  program.  Tht  successful  preparation  of  a flight  test  facility 

t 
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prior  to  the  first  launch  includes  the  following  type  of  activities: 

1.  Preparation  of  interface  specifications  and  design  criteria. 

2.  Preparation  of  layout,  installation,  and  detail  design  drawings. 

3.  Review  of  designs  by  associated  contractors. 

4.  Construction  (generally  through  subcontracts),  or 

5.  Construction  surveillance  if  actual  construction  responsibility 
rests  with  the  Corps  of  Engineers  or  another  agency. 

6.  Activation  - the  installation  of  system  peculiar  hardware. 

7.  Test  and  checkout  of  the  facility  to  insure  adequacy  prior  to 
first  launch  attempt. 

8.  Selection,  training  and  deployment  of  field  personnel. 

Since  the  preparation  of  a test  site  is  generally  a long  lead  time 
item,  early  identification  of  test  objectives,  philosophies,  system  hard- 
ware, and  auxiliary  equipment  (instrumentation,  test  and  checkout  equip- 
ment, and  special  test  tools)  will  materially  aid  in  test  facility  planning. 

RANGE  SAFETY  PLANNING 

There  is  a tendency  in  planning  a missile  system  flight  test  pro- 
gram to  neglect  or  attempt  to  ignore  the  requirement  for  a range  safety 
system.  This  generally  occurs  because  of  one  or  a combination  of  the 
following  reasons: 

1.  All  early  effort  is  concentrated  on  the  tactical  system  design; 

2.  A range  safety  system  complicates,  and  sometimes  compromises  the 
"tactical  design";  and 

3.  They  are  costly. 


Nevertheless,  because  most  test  ranges  contain  large  numbers  of  complex 
instrumentation,  extensive  facilities,  numerous  personnel,  and  are  some- 
times located  too  near  large  population  centers;  the  test  program  will 
not  be  allowed  to  operate  without  range  safety  provisions.  The  initial 
step  is  to  contact  the  designated  test  range  and  determine  the  established 
range  safety  criteria.  Based  on  this  determination  the  requirements  for 
a range  safety  system  should  be  developed  and  processed.  The  activities 
required  include  some  or  all  of  the  following: 

1.  Failure  mode  studies. 

2.  Flight  envelope  or  "footprints"  studies  for  various  malfunctions 

3.  Conceptual  design  studies  to  evaluate  methods  of  meeting  the 
range  requirements. 

4.  System  design  and  documentation.  This  should  include  sequence 
formal  reports. 

5.  System  verification,  test,  and  demonstration  including  test 
hardware. 

Most  range  safety  systems  will  require  not  only  on-board  provisions,  but 
also  special  test  and  checkout  equipment  on  the  ground  to  insure  opera- 
tional readiness  before  launch. 

RANGE  INSTRUMENTATION  PLANNING 

Data  measurements  necessary  for  system  performance  evaluation  must 
be  determined  in  the  initial  program  planning  phase.  Subsequently,  a 
determination  is  made  of  the  measurement  and  data  retrieval  systems 
available. 

Sources  for  data  measurement  and  retrieval  will  generally  fall 
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into  three  areas: 

1.  Government  operated  national  test  ranges. 

2.  Privately  operated  industrial  test  facilities. 

3.  Self-furnished  by  the  program  contractor  (prime  or  sub). 

An  overall  data  acquisition  plan  should  be  formulated  by  the  system 
test  planner  to  take  full  advantage  of  test  capability  documentation  that 
are  available  from  range  agencies  and  test  facilities. 

Most  of  the  government  operated  test  ranges  (orbital  missions  ex- 
cepted) operate  under  the  National  Range  Documentation  (NRD)  system  and 
will  provide  upon  request  NRD  publications  which  detail  the  facility 
capabilities  for  supporting  program  data  requirements.  Most  data  in- 
volving vehicle  tracking  and  position,  meteorology,  reception  of  airborne 
radio  frequency  data  including  telemetry  and  radar  safety  surveillance 
must  be  obtained  directly  from  National  Ranges  due  to  excessive  cost  to 
publish  the  document  and  security  restrictions. 

Under  certain  conditions,  "state  of  the  art"  hardware  may  be  re- 
quired for  the  collection  of  some  data.  Seldom,  if  ever,  is  such  hardware 
available  at  government  test  hanger  or  facilities.  The  planner  must  con- 
sider furnishing  additional  funding  for  the  Range  agency  to  purchase  and 
operate  program  peculiar  equipment,  (e.g.,  laser  trackers,  IR  devices, 
advanced  camera  installation,  etc.) 

TEST  RANGE  COORDINATION 

The  official  method  of  coordination  between  a contractor  system  test 
planner  and  a government  test  range  is  by  way  of  the  National  Range  Docu- 
mentation system.  Two  important  documents  that  must  be  completed  prior 


to  range  commitment  to  accommodate  a missile  test  program  are  the  Program 
Introduction  (PI)  Document  and  the  Range  Statement  of  Capability  (RSC) 
document. 

The  PI  includes  a definition  of  program  time  phased  activities  and 
is  a request  to  the  Range  to  supply  specified  range  resources  to  support 
the  test  program.  In  addition,  it  contains  that  level  of  detail  on  the 
weapon  system  characteristics  to  allow  the  Range  agency  to  determine  the 
amount  of  support  required. 

The  RSC  is  the  Range  response  to  the  Users  PI  submittal.  It  out- 
lines the  support  services  to  be  furnished  to  the  user.  The  PI  and 
approved  RSC  forms  a basic  agreement  between  the  user  and  Range  guides 
subsequent  detailed  planning  documents  for  both  User  and  Range. 


CONCLUSIONS 


This  report  afforded  the  author  an  opportunity  to  research  and 
survey  recent  changes  in  Department  of  Defense  and  Army  policies  in  the 
area  of  weapon  system  development  testing.  The  benefits  of  this  effort 
are  reflected  in  the  reorientation  of  the  author's  thoughts  on  "tailoring" 
a test  program  away  from  one  of  total  performance  consideration  and  toward 
one  with  balanced  considerations  of  cost,  schedules,  and  performance. 

Early  and  comprehensive  program  test  planning  during  the  Conceptual 
Phase  is  essential  to  program  success  in  that  it  allows  the  contractor  to 
identify  and  tradeoff  test  elements  in  order  to  surface  viable  test  pro- 
gram alternatives  and  then  select  a test  program  that  will  adequately 
evaluate  the  technical  performance  of  deliverable  hardware  within  pre- 
scribed cost,  schedule  and  performance  constraints. 

It  is  feasible  to  adopt  a modified  systems  engineering  approach  as 
a test  planning  tool  for  examining  the  interactions  between  program  test 
elements  and  all  other  program  efforts  and,  thereby,  give  the  contractor 
visability  in  identifying  and  controlling  test  efforts,  test  costs,  and 
test  cost  estimates.  This  technique  was  demonstrated  by  applying  it  to 
a hypothetical  Conceptual  Phase  (in  chapter  2);  however,  its  utility  in 
other  life  cycle  phases  is  obvious. 

The  potential  problems  that  face  the  contractor  System  Test  Manager 
as  a result  of  the  new  policy  changes  on  developmental  changes  are  many. 
One  problem  is  the  decrease  in  flexibility  to  implement  alternatives  for 
conducting  integrated  test  activities.  Traditionally,  test  cost  accounts 
have  been  the  repository  for  management  reserve  funds.  Consequently, 
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funding  of  test  activities  has  been  relatively  unencumbered  so  long  as 
a sound  technical  test  approach  was  followed.  This  cornucopia  has  been 
turned  upside  down  now  that  cost  has  replaced  technical  performance  as 
the  prime  consideration  in  the  weapon  acquisition  process.  The  System  j 

Test  Manager  must  be  vigilant  in  his  efforts  to  assure  that  the  PM  is  not  ! 

overcome  by  enthusiasm  to  comply  with  cost  objectives  to  the  extent  that  j 

i 

the  test  program  neither  provides  an  adequate  basis  for  uncovering  latent  j 
technical  risks  nor  confirms  hardware  performance  to  specified  require-  i 

ments.  The  solution  to  this  problem  can  be  accomplished  through  close 

coordination  among  the  System  Test  Manager,  the  Program  Manager  and  other 

t 

managers  of  the  program  team. 

Another  - and  in  the  author's  opinion  the  biggest  - problem  facing  j 

the  contractor  System  Test  Manager  lies  within  his  test  team  and  sub- 
ordinates. By  and  large,  test  planners  and  test  engineers  sense  the 
change  in  emphasis  of  program  objectives;  but  are  not  totally  conscious 
of  the  impact  these  new  policy  changes  will  have  on  their  day-to-day 
activities.  The  System  Test  Manager  must  make  a concerted  effort  to 
educate  his  team  in  new  program  cost  techniques  as  they  apply  to  the  test 
area  without  destroying  or  discouraging  their  diligence  toward  technical 

excellence.  He  must  teach  them  that  cost  and  technical  performance  Oh- 
s' 

jectives  are  not  dichotomous  entities,  but  rather,  are  elements  that  must 
be  blended  in  proper  proportions  to  assure  program  success. 
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